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ABSTRACT 

This paper introduces a new measure of attitude, the 
"attitude pie," which attempts to separate true neutrality from 
degree of indifference while providing an evaluation score on the 
issue being judged. A large reliability-validity study involving ten 
attitude traits, four different methods, and over one hundred 
subjects at three different times was undertaken. Results from the 
study indicated that the reliabilities of both the salience and 
positivitY scores of the attitude pies behave acceptably as compared 
to Semantic Differentials, Likarts, and Rank Importance procedures. 
Similarly the validity of the positivity scores is roughly comparable 
to that of the Semantic Differentials and Likert scales for 
evaluation. On the basis of the evidence from this study it was 
concluded that the attitude pie is at least as useful a tool in 
as:^:essing group positivity and group indifference as either Semantic 
Differential or Likert methods. (TS) 
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The various techniques of attitude measurement must presume that the true, 
underlying attitudinal construct is being represented in the observe* scores. 
However, most paper-aul-pencil measures such as the widely used Likert and Semantic 
Differential procedures avoid treating the ambiguity associated with the so-called 
••neutral" response. When the subject checks the middle response on these instruments, 
the experimenter cannot be sure if the subject is completely indifferent or i*" the 
subject has judged the issue thoroughly and is equally favorable and unfavorable. 

This paper introduces a new measure of attitude which purports to separate true 
neutrality from degree of indifference while providing an evaluation score (which 
we call "positivity") on the issue being judged. Thus the procedure provides a measure 
of affective favorability and unfavorability on the issue, as do the more standard 
paper and pencil techniques, as well as a measure of indifference \^ich purports 
to separate those who are apathetic on the issue from those who are truly neutral. 
It is this latter measure V7hich is the key point of difference between what we call 
"attitude pies" and the more typical linear scales exemplified by Likert and Semantic 
Differential instruments. 

With the typical linear scales respondents are asked to indicate their feelings 
about attitude objects by simply marking a single space along a continuum. The 
bi-polar scale used with Seuiantic Differential and Likert scales is replaced by a 
circle or 1 3/8" "pie" with a dot placed in the center: 




Subjects arc told to consider the pie as representative of "all your feelings" 
about the attitude object or concept in question. They are given instructions to 
respond by drawing in three "slices" on the pie to represent the degree of 
positive, negative and indifferent components of their attitude toward the object. 
The slices drax^ in, then, are thought to reflect relative amounts of positive and 
negative affective regard for the attitude object as well as the degree of indiffer- 
ence associated with the object. A typical pie response with rtughly equal positive 
and negative affect, and with a good slice of indifference would appear as: 




Positive, negative and degree of indifference scores are derived by employing 
a protractor to measure the degree of arc subtending each of the sectors. Since 
the sum of the angles in a circle must equal 360 degrees, only two angles need be 
measured to generate scores for the three attitudinal components. The scores that 
result show relative amounts of positive and negative feelings toward an object or 
concept as well as the overall intensity of these responses. 
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By measuring the angles of the various slice??, four scores are rt<erived. The 
positive score is simply the size of the angle representing that component of the 
respondent's favorability, t^ie '^plus^' slice* Similarly, the nesative score is the 
size of the angle of the "minus'' slice and is presumed to be the subject's unfavor- 
ability on the issue. Positivity is the positive score minus the negative score and 
obviously it csw range from -360 to +360* In the example immediately below the 
positive score (150) minus the negative score (80) produces a positivity score of 
70. This yields a single evaluative score for each attitude object vhich is com- 
parable to the scores generated by Likert and Semantic Differential scales. 



+ « 150 
- = 80 
Positivity « +70 
Indifference » 13P 



Indifference scores are derived by subtracting the sum of the positive and negative 
scores from 360. 

The analysis of indifference scores allows the researcher to categorize posi- 
tivity scores into classes or along a continuum. For example, in the three pies 
below the positivity is zero for each but the degree of indifference is decreasing 
from left to right. The qucjitior; that every research who has used Likert or Semantic 
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Differential should now be asking is "How would people who are equally positive and 
negative but more or less indifferent respond on the standard scales?" Similarly 
are individuals who are slightly positive (or negative) and highly indifferent more 
likely to check the midpoint on a Semantic Differential or Likert than individuals 
with the same positivity j;core but with less indifference? 

The reasons that responses to these questions do not immediately come to mind 
reflects the fundamental ambiguity of the midpoint on Semantic and Likert scales. 
Quite simply, wn are arguing that to disambiguate the midpoint of the Semantic or 
Likert scales requires at. least two pieces of infortration, one related to positivity 
of affect and one related to indifference or apathy (or its complement salience). 
In principle, the pies provide this information. Whether they do so in practice is 
the overriding empirical question. 

A part of the empir:lcal question is whether subjects will provide both in- 
difference and positivity information on the pies. Earlier research with the pies 
showed that individuals can and do use the pies to show feelings of indifference and 
positive nnd negative affective reactions tov/ard a variety of attitude objects 
(Lull and Arntson, 1974). In a sample of 41 persons who evaluated teu attitude objects 
nearly everyone (97.57.) reflected a component of "indifference" at least once. Ninety 
percent of the sample used the indifference component in combination with positive 

4 
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and negative slices on at icnst one of the Itcmc. Subjects were also likely (70.7%) 
to mark a pie as cither all fOsitivo or all negative on at least one of the ten 
objects. Thus people seem .rilling to use the range of responses which the pies 
provide* 

In the following pages X7e wish to argue both the theoretic and practical 
advantages and disadvantages of the pies, describe a study designed to test the 
reliability and validity of the pies versus Likert-typc and Semantic Differential 
competitors, and present data from that study which is both favorable and un- 
favorable to the pie instrument. Let us first take up the advantages and disadvan- 
tages of pie measures both in substantive applications and as a measurement tool. 

Tim PROS AND CONS OF PIES 

There are at least three areas of current theory and research on attitudes that 
could be illuminated by employing pie-type measurement procedures. First in the 
area of interpersonal attraction and balance recent work has been careful to 
distinguish between pairs of people who are neutral toward one another and pairs 
who are indifferent to one anothci:. As Hunter (1974, pp. 26-34) points out the 
various theories of how people group in sociometric networks (balance, positive 
balance, clustering, and hifi o\m dynamic invariante) have different predictions de- 
pending upon how "non-indifferant" links are treated. Of course indifferent linkb 
must be able to be separated from those which are not if theories are to be tested. 
Th« f.ios provide a convenient raethod for doing this. In a similar vein Cartwright 
and Harary (1972) have argued that the generalization of structural balance theories 
to networks "provides a meaii' for representing more realistically the full range of 
evaluative relationships [among people] that are encountered empirically" (p. 498). 
In order to achieve this goal they introduce at a theoretical level the value, or 
measure, of a directed lir^o as the ordered pair, p for positive, and n for negative, 
where p represents positive attraction to another and n disliking. Their triangle 
model which introduces a positivity score, p-n, an intensity score, p+n, and the 
condition that p^-n is bounded by 1, is quite conveniently tested with the pie in- 
strument. Balance theories and their e:ctensions are cognizant of and in need of 
measurement procedures w^'^ich the pies readily provide. 

Secondly, in the attitude formation and change area there is some gro:7ing 
intere<3t (especially by Lashbrook and his associates: Hylton and Lashbrook, 1972; 
L:*.'hbrook tnd Bush, 1973; Lashbrook and Sullivan 1973; Sullivan and Lashbrook, i074; 
Troyke anJ L^^shbrook, 1973; W. Lashbrook and V. Lashbrook, 1973) in the bchavlo;: of 
audiences which are apathetic as opposed to those X7hich are neutral. Apathctics and 
neutrals nave been distinguished using the Fishbein and Raven (1962) attitude and 
belief scales, salicncy scales, and expenditure of effort scales (as cited by W. 
Lashbrook and V. La*5hbroolc, 1975). The pie measure can provide the necessary 
classifLcations as a standard part of the informr.::ion provided by an audience in their 
favor al ility rotjng3 rather than as information ovoc and above that provided by 
favc^vability scores. 

Tl Irvlly, tho pie tcchidoue offers some intuitively satisfying information to 
lay consuTTier.' ol /(^search on public attitudes. lo a study currently being carried 
out by one oT uhc authors on oudlence rccoanition and favorability toward regional 
radio stations, the pies arc providing both types of information simultaneously and 
in a form which is readily crmprehensiblc to radio station managers. In particular 
l.he iu-Urforcnce score is roadll^ associateu with lack of recognition ana the 
positivity ycore with favorability. Placing average pies for various demographic 
groupings of the audience side-by-side facilitates the comparison of subgroups without 
^ recourse to numerical (usually percentage in this case) summaries. While not directly 
pnj^" pertinent to theory or theory-related research this dimension may serve as the 
LIML strongest reconuaendation for the pies to the broader audience. ^ 



Despite the above de.i.jr^od advantages, there are drawbacks to the pie instru- 
mentation. Firsu, the pL:. ..occssitrte that subjects make judgments which are less 
familiar to them than judgments made tn the more common 5-point Likert-type and 
7-point Semantic DificrentlaJ. scales. VhilQ people often see for example, the "tax 
dollar" sliced in neX7S descriptions of government spending and, hence, find it familiar 
enough to comprehend they seldom are asked to make judgments which slice their own 
attitude pies. This lack of familiarity will be likely to increase the piel's un- 
reliability. Besides asking for unfamiliar types of judgments the pies require 
more complex judgments. The increased complexity can be seen in two ways: (1) Three 
types of information (although only 2 are independent) are required by each pie 
judgment--poslcivG sentiment, negative sentiment, and indif ference--as opposed to 
the one dimensional judgment of the Likert and Semantic Differential techniques. 
(2) The subjects express judgments on a circular continuum with no set number of cat- 
egories. Discriminations by subjects can be as fine as they wish (although our 
protractors usually fix a lower limit of a 5 degree slice). While finer discrimina- 
tions permit more subtle differences, they are also likely to increase unreliability. 
15y standard criteria for measurement reliability (Torgerson, 195 , p. ).^ the 
pies are likely to exhibit greater unreliability than the simpler and more familiar 
Likert and Semantic Differential instruments. While the degree of difference in 
reliability bctx^een pies and competitors is an empirical matter, we hope that the 
theoretic advantages of the pies are an acceptable trade off for a tolerable decrease 
in reliability. 

In this section x^e h&ve argued for the potential advantages of the pj.t instru- 
mentation over its more acr.(.:ptcd counterparts. Now x^e must subject our argtarjsnts to 
the critical scrutiny of cuip.'rical test. In particular a study of the reliability 
and Lritemal validity ot the pic.3 was undertaken against the competitive techniques 
of Likert scaling and Semantic differential scaling. These competitors were chosen 
because of the extent of their use by researchers and because a recent study 
(Jaccard, \Jabc*: and Landmark, 1975) has shovm that these two techniques arc highly 
reliable over time (with test-rctest correlations all above .80) ancl that they 
exhibit considerable convergent validity as compared to fancier Guttrflan and Thurstone 
scal'^s- Our choice of Likert and Semantic Diff .^rential scales as baseline compari- 
sonr, for the reliability and internal validity ol the pies should bias all of our 
conclusions abr.ut the pies tox^ard the conservr^tivc. 

TESTKG THE ATTITUDE PIES 

Tno toptir.g procedure was a variant of the basic multitrdt-multimethod (MTMM) 
technique advcnced by Campbell and Fiske (1959). The traits included 10 difiTerent 
attitude itcm:>. The methods included 7-point Likert-type scales, 7-point Semantic 
Differential scales, and attitude pies. The variation from straight VHMl proce- 
dvirer simply involved three x:aves of data gathered on the caine subject samples rather 
than oiie. The availability of overtime data maken it possible to check uhe cest- 
retost- reliability of the methods and to check for loarning effects (inct cased 
familiarity) with the pies. 

TRAITS AND 14ETH0DS 

Based on pvov»ou.^ research Lull and ^l:ntson (1974) 10 attitude items \7eTa 
chosen as traits. They wero cacJcn to maximiicre variance across trails jv indifference 
and positivity *?cores. Thus "nlka-seltzer , " '^radishes," and "polkas" were expected 
to be largo Indifference it3Trs, "Parents," "making love," and "inflatica" x;ere 
expectod to ivj low on indifference and "isummertimo," "thirties," "giving blood." ond 
"handguns" were expected to be of intermediate value on indifference. 



Each of the ten traits was evaluated by the subjects V7ith at least two of the 
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methods at each point in I'^.^r-^ At the third administration all three methods were 
used by all subjects^ Tho iastruction set for the attitude pies is included as an 
appendix to this paper • Thevo is ano pie co be sliced for each trait • For the 
Semantic Differential fi\3 scales for each trait were used. Four adjective 
antonym pairs were constant across the 10 traits: good-bad, positive-negative,^ 
meaningful -meaningless, nnd important-unimportant • Osgood, Suci, and Tannenbaum 
(1957, p. ), report that oil these pairs load cleanly on the evaluative factor. 
The latter tx;o pairs x/ere chosen because they were meant to tap a salience dimension 
for each trait. The supposition we made was that salience or importance would be 
inversely rclctcd to the indifference score of the pics. The fifth adjective antonym 
pair was chon^n so that it loaded on the evaluative factor but was conceptually 
relevant to the trait being evaluated. Thus the pair "kind-cruel*' is relevant to the 
tvait "parents*' but irrelevant to "polkas." 'H^orthlcss -valuable" is relevant to 
' :^olIcas" but much less relevant to "parents/' This technique Is suggested by 
Osgood, Suci, and Tannenbaum (1957, p. )• 

Parallel to the Semantic Differential procedure five 7-point agree-disagree 
scales constituted the Likert-typc items. The sentences which subjects were to 
dlG.ngrcc with or agree x7ith (along the 7-point: continuum) were constructed to be as 
semantical ly close to the Senantic Differential adjective pairs as possible. Thus 
two 3ii5itGnces of the five V7ere designed to tap importance and meaningfulness . The 
other sentences sought to tap overall favorability-unfavorability for each trait, 
for cxaraplc: "Vlhen I dance, I hate to polka" or "Summertime is pure joy." Further- 
mov^, the 7 agree-disagrot. CiiuGgories for Likerts were structured identically to the 
7 categories of the Semantic Differential adjectives. Here again the similar 
cir.r-icturc of the instruments should favor the consistency between the Scmantx-^^ 
Differential and Likerts rather than either of these and the pies. In order that 
tho reader may conN/ince himself or herself of the parallelism of the Semantic 
Differentials and Likerts, &n the Likert sentences and the unique Semantic 
Differential adjective for each trait are reported in Appendix B. 

ADMINISTRATION PROCEDURES 

^.1 -.he spring trx-^. of 1975 questionnaire wore administered lo an initial sample 
of slightly more than 100 undergraduate students in an introductory communication 
course at the University of Uioconsin. The students vyere distributed across Q dis- 
cussion sections of approximately 15-20 students per section. Three of the discussion 
flectioas wore randomly assigned to one administrative group, Group A. The other three 
were assignee to Group B. The groups were approximately equal in size, 

Gropp A received three adiidnistrations of the attitude questionviaire separated 
by on? month delays. The first questionnaire asked Group A subjects to (1) to 
rank the attitude items in terms of importance, (2) to evaluate the items on pie 
5?c*3les, and (3) ;:o evaluate the items on the previously described Semantic 
HU.-^rc-afial scales. The second administration of the qlle3^ionnaire was identical 
to the first. The third oue«3tionnaire was identical to the first two except that 
'chz previously described Likert scales were added. 

Group h rcceivoci the i^r^o treatment as Group A except they received the Likert 
jcales and nof the Semantic Differential scalos at times 1 and Z. As with Group A 
the scale which did not appear at times 1 and ?. did appear at time 3i Table 1 
summari7,es tho administration procedures and associated sample sizes. 
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Although some subje^.. a'rt'iition occurred between tivnes, it vas not appreciable. 
Between times 1 and 2 Group A has a sample size of 51 and Group B of 53; for times 

2 and 3 sample size is for Group A and 46 for Group B; between times 1 and 3 
there arc 49 subjects with data in Group A &nd 46 in Group B. Across all 3 times 
there are 46 subjects in Group A and A3 in Group R. 

SL'MMARIZING THE DATA 

In ordar to treat the data in mm terms, various indices were constructed. 
The rank order data remained the same with highest numbers indicating lowest rank. 
The pies were transformed into a positivity score (range -360 to +360) and on 
indifference score (range 0 to 360). The three purely evaluative items are the 
.Semantic Differential V7ere summed to yield an evaluative score (SDE) (range 3 to 21). 
The two other items were also summed to yield a salience score (SDS) (rrnge 2 to 14). 
fo-'ilar indices were constructed for the Likert items (LE and LS) with the same 
ranges as the SDE and SDS indices. 

Consider first the evaluation indices. Group A has 10 traits and 2 methods 
(SDE and positivity) at time 1, 10 traits and 2 methods at time 2, and 10 traits and 

3 mo':hods (LE Is added) at time 3. Group B has the same number of traits and methods 
n.- the i.urc-.c times but the LiUert method switches roles with the Semantic Differential 
method. Ibus, there are 3 ITiKrVs to study (one for each time) for each group. 

N-xi- consider the sali.enr.e and indifference indices. Group A has 10 traits 
ana 3 methods (indifference, r/wiks, and SDS) at time 1, 10 traits and 3 methods at 
time 2, and 10 traits and 4 w^thods (LS is added) at time 3. Group B ha.-: the same 
uuirDor of traits and methods uith the Likerts and Semantic Differentials switching 
roles. Thus, there are 3 HTMM's to analyze (one for each time) for each group. 

Obviously this tremendous volume of data must be reduced to a manageable form 
<ind at the same time reveal the subtleties of the pie's performance relative to that 
oP the Likerts and Semantic Differentials. Ug turn to this task in the next section. 
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RDLIABILITY OF THE PIES 



Xhere are relatively few techniques available to the researcher for the purposes 
of estimating the reliability and validity of proposed measures. The reliability 
of a technique can be assessed either through its internal consistency (assuming 
parallel tests) or its consistency overtime (test-retest correlation). The validity 
of a measurement procedure can be evaluated in. terms of its internal validity'- 
(correlation with another procedure presumably measuring the same trait) or predictive 
validity (correlation with an effect which is theoretically caused by the variable 
as measured by the technique under question). We restrict our evaluation of attitude 
pies to their over-time reliability and to their internal validity. 

Out goal in the rerainder of this paper is two-fold: (1) to present data on the 
reliability and validity of the pies and (2) to assess the relative meiits of diff- 
acent techniques for determining reliability and validity of the pies. In order to 
accomplish both goals we shall very carefully describe the assumptions of each 
technique used and evaluate those assumptions relative to our own data. The des- 
cription of assumptions will be facilitated by employing path diagrams (Heise, 1968; 
Duncan, 1971) whenever possible. 

TEST-rxKCEST RELIABILITY 

Perhaps the most time honored method of assessing over- time reliability is the 
straightforward test-retcst correlation. Figure 1 represents the case of a single 
variable measured at two points in time. X* is the true value of the variable and 

-^.ts observed counterpart. e(l) and e(2) are random errors of measurement at each 
o'- che two time points. u(2) represents variables which affect or disturb X daring 
(.hvi time interval which has elapsed (Heise, 1971, p. 352). As can be seen from 
the path diagram, this measurement model assumes that the measurement errors are 



Figure 1 Here 



uncorrelated with true scores, that disturbances are uncorrected with exogenous 
variables, and that the reliability p is stable over time (Heise, 1971, pp. 352-53). 
As can be easily seen from the diagram rj^2 ~ P P21 where ^® ^^^^ 

correlation between X(l) and v(2). This equation cannot be solved Unless some 
assunptions about made. Since is a measure of the stability of the true 

scores over time, it is often assumed that P21 = 1 implying perfect stability. Under 
these assumptions the reliability p^ is just Vii. 

If wo were to view the data on each trait as if it were gathered independently 
over the panels time-1 and time-2, and time-2 and time-3, then by the model of 
figure 1 the reliabilities would just be the test-retest correlations. These are 
reported in Table 2. 



Table 2 



With the sample sizes as reported in Table 1, all the test-retest correlations 
reported in Table 2 are statistically significant at p < .025 at least. Even though g 
r% all the reliabilities (except rankings for group B) show more consistency between 
FRir" times 2 and 3 than between times 1 and 2, none of these increases is statistically 
£iML significant by the z-test (Hays, 1963, p. 532). These small learning effects are not 
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appreciablc. Thus incie..Ke>- familiarity with the pies does not produce appreciable 
increases in test-retest ccr.sf.stcn cy. 

When the test-retest correlations for the pies are compared to those for the 
Semantic Differentials, Llkerts, or Ranks, the pies invariably perform less con- 
sistently on both the salience and evaluation dimensions. However, although the 
differences in reliability favor the more standard measures none of the differences 
achieve statistical significance. The lack of statistically significant differences 
is an Important matter since we must be concerned not with the absolute reliabilities 
of the pics but with their reliabilities as compared to other standard techniques 
under the same testing procedures. The absolute reliabilities of the pies are not 
as high as most researchers would like (the .7 to .9 range being more acceptable) 
but by the same token the reliability of the Semantic Differentials and Likerts i« 
below that reported in the Jaccard, Weber, and Lundmark (1975) study. 

Part of the reason for this depreciation in consistency over time can be 
attributed to the longer time delays in administering the retest questionnaires 
(1 month between administration) then is often allowed in research settings. With 
shorter time delays an increase in the unreliability of any scale is to be expected. 
A second reason for low test-retest correlations is that the test-retest correlation 
version of reliability assumes that the true scores are perfectly stable across 
time. With some of the traits employed, this assumed stability is highly unrealistic, 
esbecially with the long time delays between measurements. For example, 'Inflation, 
was a national issue during the Winter and Spring of 1975 as the country slowly move., 
from a very unstable economic situation to an increasingly stabilized one. Similarly, 
"parents," "suttmertime" 'making love," and "parties" could be dramatically affected 
by the information and e.:periences of a two month period. The point is that 
reliability should not be confounded with stability. A measure of reliability which 
separates out stability ia needed. We consider such a measure next. 

RELIABILITY INDEPENDENT OF STABILITY 

In the diagram of Fipure 1 we saw that in order to treat a test-retest correla- 
♦•ion a'? a re] ^'aiiility it was necessary to assume that the stability between true 
scores' at times 1 and 2, Po^, was perfect, or equal to 1. In other words, the situa- 
tion is underidentified without further information. As Heise (1971) points out, 
if three wsv.es of data are present the underidentified model of Figure 1 becomes 
identified and "a new measure of reliability based on test-retest data but free of 
temporal change effects" (p. 355) can be developed. Figure 2 describes the 3-wave 
situation and the associate assumptions. The assumptions for this case are quite 



Figure 2 Here 



similar to the case of Figure 1. In particular note that (1) the stability of the 
true scores P2i and p32 i° assumed to be the same between panels, (2) the errors 
oc measurOTG.it are mutually uncorrelated so that test-retest sensitization is ?ssumed 
negligible, and (3) the reliability, p^, is assumed constant across time intervals. 

With these assumptions it is an easy matter to show that a measure of the 
reliability which is independent of temporal change (Heise, 1971, pp. 354-55) is given 
by p = r;^2'^23'''^13« advantage of this measure is that it does not, assume 

anything about the degree or sign of the stability of the true scores between each of 
the 5 waves of data. It does, of course, assume that the rate of stability or 
instability is constant. On the other hand, this measure has the disadvantage of 
O allov;ing peculiar values for reliability. The p2 value above can be greater tha'i 1 
ERIC or can be negative. In both cases, the parameter value is uninterpretable and cannot 



ii 



-9- 

be used. Helse does not cou.ment on this mat^-.er. 

A model similar to Heise's was developed by Wiley and Wiley (1971). The Wiley 
and Wiley approach does not necessitate the assumption that the reliability is con- 
stant across time intervals and does not necessitate the use of standardized data. 
It does however assume that error variance is constant across time. Since the 
Helse reliability can be shown to be identical to the Wiley and Wiley reliability 
at time 2, then there is no advantage to the Heise scheme. In the tables which 
follow only the Wiley and Wiley indices of reliability will be reported. The 
reader should keep in mind the identification between the reliability at time 2 
and Heise's measure. Wiley and Wiley also insist upon the use of unstandardized 
measures. Ue will follow suit allowing the variance inherent in each measurement 
method to influence the measures of reliability for each technique. 

In Tables 3 and 4 Wiley and Wiley's measures of reliability (1971, p. 368) 
averaged across the 10 traits are reported for both groups at the 3 time periods. 
Table 3 reports the reliabilities of the evaluation measures and Table 4 reports 
reliabilities for the salience measures. 



Tables 3 and 4 



The positlvity and indifference scores using the Wiley and Wiley indices for relia- 
bility are still somewhat smaller than the reliabilities of competitive scales within 
a given group and within a given time frame. However, the differences are not sig- 
nificant, averaging less than .1. 

When the reliabilities from the 3-wave constant error variance model are com- 
pared to reliabilities from test-retest correlations (Table 2), three observations 
should be noted (1) overall the reliabilities on all scales are greater in the 
former case. (2) The pie reliabilities are less different from competitive reliabil- 
ities in the Wiley and Wiley model than in the test-retest model. (3) There is a 
non-significant but consistent trend toward greater reliability for the evaluation 
scalos in boch the Wiley and Wiley and the test-retest cases. While there is no 
way of testing which of the two models better fits the data (since both are exactly 
identified), we feel that there are obvious reasons to favor the Wiley and Wiley 
constant error variance model over the earlier one. Wben one considers the kind of 
traits employed in this study and the time logs between measurements then (to assume 
that true scores are perfectly stable across time) seems a mere convenience at beat. 
If the Wiley and Wiley indices of reliability are to be preferred, then it is only 
fair to conclude that the pies both in the evaluative and salience dimensions are 
not appreciably less reliable than standard Semantic Differential, Likert, and Rank- 
order methods of assessing favorabillty and importance. 

CORRELATED ERROR MODEL 

Not being satisfied with the pies as not appreciably less reliable," we scught 
to exploit other nofcativo characteristics of Semantic differential, Likert and Rank- 
order scales. In i;articular, the simplicity of Judgments required by these proce- 
dures is both a boon and bane. On the positive side, simplicity of judgment helps 
to Insure reliability or consistency. On the negative side, simplicity of judgment 
may enhance the liklihood of test-retest sensitization whereas the more complex 
judgments required by the pies may be less susceptible to sensitization. If this 
reasoning is accurate, then by including curved arrows (indicating correlation) 
among the errors of measurement e(l), e(2), and o(3), in Figure 2 and solving for the 
O reliability, then test-retest sensitization might be accounted for. 
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Unfortunately, such a modified model is severely under identified as Heise 
(1971, p- 358) notes. Fortunately, another set of Wileys (Wiley and Wiley, 1974) 
solved the underidentification for any 3-wave equal time lag data set by postulating 
a first-order autoregressive scheme among the errors. Unfortunately, the reliability 
indices for the correlated error model do not produce interpretable results for the 
data set of this study. 

While this lack of interpretable reliability values is frustrating and incom- 
prehensible to us, it is not totally unexpected since as Wiley and Wiley (1974, 
p. 105) point out: "The correlated error model does not yield admissable parameter 
estimates for all sets of data/' Using the correlated error model algorithum reported 
by Hilcy and Wiley (1974, p. 181) does not yield a single reliability estimate which 
is less than 1. All estimates fall in the range 1,0 to 2.2. Obviously, there are 
some conditions under which the correlated error model does not hold. Unfortunately, 
VJiley and Wiley have not specified when uninterpretable results are to be expected. 
Whatever these conditions are, this study satisfies them perfectly. Thus we must 
be satisfied with the reliabilities from the constant error variance case and live 
with the confound of sensitization. 

VALIDITY OF THE PIES 

Attitude pies offer two types of information: evaluation and salience. In order 
to establish the validity of the pies, validity of both the positivity scores and the 
evaluation scores must be established. This section will present data on the 
validity of the pies prinsrily through the multi-trait - multimcthod technique 
(Campbell and Fiske, 1959). 

HTMM MATRICES FOR EVALUATION SCORES 

Referring back to Table 1, it can be seen that there are 6 potential KTMM matrice 
to be displayed: 3 for Group A and 3 for Group B. Rather than displaying the 1430 
correlations from these sir. matrices we shall marely present the validity diagonals 
r:>/oragcd across the appropriate groups and times. Figure 3 suirmarizcs the structure 
the averaged >5TMM. 



Figure 3 Here 



As •':ho figure indicates not all the monomethod-hctcrotrait blocks and heteromethod- 
;»-cerotrait blocks are based on the same number of observations due to the asymmetry 
of the questionnaire administrations overtime. 

The single most important portion of this matrix is the validity diagonal of 
eacii heteroir ^nhod block. The averaged diasonals for each heteromethod block of 
Figure 3 are presented iu Table 5. The correlations reported are averaged across 



Table 5 Here 



groups end times as indicated in Figure 3. Campbell and Fiske 's first criterion for 
validity (1959, p. 82), namely that "entries in the validity diagonal should be 
O significantly different from zero and sufficiently large to encourage further exam- 
FRIC ination of validity ''is clearly met for 9 of the 10 traits in table 5. Only ^'infla- 
aaa™ tion'» does not exhibit the desired validity* This anomalous result is rather ^2 
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befuddling and inexplicable to us. Aside from this single trait Campbell and Fiske s 
first and most important criterion is easily met by the positivity scores. As with 
the reliability measures previously, the scales competing with the pies tend to 
outperform them. The validity correlations between SDE and LE are stronger than 
between either of these scales and the pies for all traits except Interestingly 
the highest intercorrelation of SDE and IS scales occur for the traits "guns, 
"blood" "polkas," and "radishes" which over the entire sample are ranked in Impor- 
tance 6th, 8th, 7th, and 9th respectively. For certain items which have been ranked 
in the upper 5 in importance, namely "parents," (#1) "love," (#3) and "parties, 
(#4), correlations between pies and SDE, and pies and LE are acre simil^ir.' to corr- 
elations between SDE and "LB scales. Thus, in this data set when the pici fail to 
exhibit the same degree of convergent validity as Semantic Differentia^ or Likert 
methods, it can often be explained in terms of the lower salience of the trait to 
the subjects. With respect to convergent validity then the pies perform about as 
well as that (that is, within .1 on the average) or better than Semantic Differen- 
tials and Likerts for 3 of the 5 most salient traits and 3 of the 5 least salient 
traits. The pies are clearly outperformed on "summer" (#2), "inflation" (#5), 
"guns," (#8), and "radishes" (#9). 

MEMM MATRICES FOR SALIENCE SCORES 

As with the evaluation scores, there are 6 OTMM matrices to be considered. In 
this case, however, averaging over Groups A and B for times 1, 2, and 3 produces a 
10 trait by 4 method (pies, SDS, LS, and Ranks) matrix. As before we will apt not 
to present the averaged MTMM of 780 correlations but offer the six validity diagonals 
for each of the ten traits in Table 6. These are enough. Of all 6 validity diagonals 



Table 6 Here 
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only the Semantic Differential and Likert methods show any pattern even vaguely 
resembling convergent validity. The highe- correlations arc somewhat misleading 
however sinco the "meaningful-meaningless" ind important-unimportant' adjective 
pairs -which foi-mcd the SDS scale and were the basis for the LS agree-disagree state- 
ments were chodcn because they loaded on the evaluative dimension in Osgood, Suci, 
and Tannebaum's 1957 volume. Thus there is some question as to whether these 
items can be said to tap salience primarily. In fact when SDS and LS indices are ^ 
correlated with rank importance they cannot be said to satisfy Campbell and Fiske s 
Erst criterion for convergent validity cited above. 

Of course the behavior of pie salfcnce scores is not very encouraging either. 
The pies do not correlate significantly with questions on importance (SDS and LS) 
and do not coridate significantly with rank importance. Despite the inability of 
any of these measures to conclusively show convergent validity on salience that does 
not exempt -.he pies from doubt concerning the validity of the indifference score as 
a measure of salience. 

Some of this aoubt can be disspelled by computing mean salience scores for each 
trait on pie, ranks, and SDS for Group A across each of the 3 times and mean salience 
scores for each trait on pics , ranks , and LS for Group B across each of the 3 times . 
By rerjktng the traits on the basis of their mean salience as determined by each of 
the three methods, then Spearman rank correlations can be calculated. The mean 
salience scores for Groups A and B are presented with the corresponding rankings in 
Table 7. The Spearman r for pies and rank.n in Group A is .964 and for SDS and ranks 
is .079. The Spearman r for pies and ranks in Group B is .916 and for rank's and LS 
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Table 7 Here 



Is .867. Thus, although <jg have no evidence that the salience score on pies can 
successfully discriminate an apathetic from a coinmitted individual on an issue, there 
is some Bvidence that groups can be discriminated in terms of their apathy or commit- 
ment on an issue. 

CONCLUSIONS AND DIRECTIONS 

This paper has introduced a measurement tool for attitudes which attempts to 
separate Individuals who are neutral on an issue from individuals who are apathetic. 
The tool seeks to accomplish this goal by simultaneously requiring individuals to 
provide information on their favorabiUty toward an issue and their relative intensity 
of positive and negative affect toward the issue. In order to establish this tool 
as an effective Instrument for research its reliability and validity in both the 
positivity and indifference components need to be established. A large reliability- 
validity study involving 10 attitude traits, 4 different methods, and over 100 
subjects at 3 points in time was undetaken. 

Results from our study indicate that the reliabilities of both the «allence and 
positivity scores of the pies behave acceptably as compared to Semantic Differentlds , 
iikerts, and Rank Importance procedures. Similarly the validity P°J^^^^^*=y 
scores is roughly compar^ible to that of the Semantic Differential and Likert scales 
for evaluation. However the SDE and LE scales do show some validity success beyond 
that of the oositivity scores for certain traits. Unfortunately, validity data on 
the salience" scores for the pies was very unfavorable although this evidence was 
mitigated by a failure to establish any mono-trait - heteromethod correlations as 
an established baseline for assessing convergent validity on the ^^li^^^^Jf^^^^J;;* 
Vlhen we considered the validity of pie measures of salience compared to rank import 
ance for grocp means rather than individual scores, the indifference scores behaved 
admirably. 

On the basis of the evidence from this study we can safely conclude that the 
attitude pie is at least useful a tool in assessing grouE po<?itivity and groifi. 
Indifference as either Semantic Differential or Likert methods. When the mean scores 
of groups are not the mode of analysis but individual scores are of prime concern, 
then the pies should provide estimates of positivity which are only a little less 
reliable and valid than Semantic Differential and Likert scales. However, the 
researcher should be hesitant to employ the indifference score when individual data 
are the prime requirement. 

It is our hope that attitude pie will be adopted in the day to day work of 
researchers ao that further data on predictive validity and reliability cau be ac- 
quired. We feel and have tried to argue that there are some , 
beyond those afforc\(.d by st.idard linear measures. Obviously, some difficulties exist 
with pie measures and these nust either be rectified by employing different instruc 
tion sets or must be more definitively tested so that the doubts about the indiff- 
erence score may be either confirmed or laid to rest. 
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TABLE I 



Administration procedures indicating scales 
given to each group at each point in time 
along with sample size 



Group A (Size) 

Rank 
Pies 

Differential 
Likerts 



TDffi-l 


TIME -2 


TEffi-3 


56 


53 


50 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


Yes 


No 


No 


Yes 



Group B (Size) 

Rank 
Pies 

Differential 
Likerts 



54 

Yes 
Yes 
No 
Yes 



53 

Yes 
Yes 
No 
Yes 



47 

Yes 
Yes 
Yes 
Yes 



X*(l) 
X(l) 

f 

e(l) 



I'm 



U(2) 

i 

-f x*(2) 

X(2) 

I 

e(2) 



Figure 1. Path model for test-retest correlations adapted from Heise (1971). 
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Table 2. Test-rctest corrcLitions averaged across traits for evaluation scores 
and salience scores. 



Time 1-Time 2 



Time 2 -Time 3 



Group A 

Positiv.Luy 

SDE 
Group B 

Positivity 

LE 

Average Positivity 



Group A 

Indifference 

SDS 

Ranks 
Group B 

Indifference 

LS 

Ranks 

Average Indifference 
Average Rank 



EVALUATION RELIABILITIES 

.597 
.704 

.577 
.759 

.587 

SALIENCE RELIABILITIES 

.486 
.576 
.600 

.309 
.653 
.632 

.401 
.615 



.668 
.760 

.658 
.782 

.663 



.538 
.612 
.656 

.554 
.705 
,598 

.546 
.628 



u(2) 



u(3) 



X*(l) 
X(l) 

I 

e(l) 



P21 



-> XA(2) 
P 

f 

X(2) 



e(2) 



P32 



X*(3) 
F 

X(3) 



e(3) 



Figure 2. Path model for the 3-wave reliability model which separates true 
score stability from reliability (cf. Heise 1971). 
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Tabic 3. Reliabilities averaged across traits for the evaluation measures using 
Wiley and Wiley's constant error variance model. 



Tiiae 1 Time 2 Time 3 

Group A 



Positivity .692 .704 .802 

SDE .753 .760 .860 
Group B 

Positivity .658 .677 .793 

LE .748 .852 .916 



Table 4. Reliabilities averaged across traits for the salience measures using 
Wiley and Wiley's constant error variance model. 



Time 1 



Time 2 



Time 3 



Group A 

Indifference 

SDS 

Ranks 
Group B 

Indifference 

LS 

Ranks 



.555 
.625 
.661 

.591 
.705 
.621 



.537 
.572 
.551 

.475 
.640 
.638 



.727 
.728 
.763 

.720 
.787 
.775 



Table 5. Validity diagonals for evaluative scores averaged across groups and 
times for pies, Semantic Differentials, and Likertscales. 





Pies vs SDE 


Pies vs LE 


SDE vs. LE 


TRAITS (Rank Impt.) 






.697 


Parents (i) 


.774 


.582 


Blood (6) 


.676 


.691 


.760 


Inflation (5) 


.216 


.152 


.688 


Guns (8) 


.648 


.659 


.804 


Polkas (7) 


.713 


.688 


.805 


Parties (4) 


.657 


.659 


.695 


Radishes (9) 


.667 


.577 


.852 


Love (3) 


.595 


.679 


.574 


Summer (2) 


.456 


.485 


.671 


Alka-Seltzer (10) 


.602 


.608 


.672 


Average 


.616 


.592 


.732 
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PIES 



PIES 



SDE 



LE 



Group A Sit 
T1,T2,T3 plus^ 
Group B at 11,12,^ 
T3 = 6 groups 



Group A at Ti,T2, 
T3 plus Group B at 
T3 « 4 groups 


Group' 

at T1,T2,T3\^ 
plus Group B ac^^-s. 
T3 = 4 groups 




Group A at T3 plus ^ 


i Group A at T 3 


Group A^\s^^ 


Group B at Tl, T2, ] 


[ plus Group B at 


T3 = 4 groups 


T3 = 2 groups 


at T3 plus^v,^^ 




Group B at Tl,T^>v^^^ 






T3 = 4 eroups 



Figure 3. MTMM matrix for evaluation scores averaged across groups and times 
as indicated ia each block: Three methods by ten traits. 
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6 Validity diagonals for salience scores averaged across groups and times 
for pies, SDS, L'2, and Ranks. 





Pies 


PIES 


SDS 


PIES 


SDS 


LS 




vs 


vs 


vs 


vs 


vs 


vs 




SDS 


LS 


LS 


Ranks 


Ranks 


Ranks 


TRAITS 














Parents 


.297 


.214 


.816 


.187 


.566 


.624 


Blood 


.453 


.297 


.482 


184 


SI 1 


A A 1 


Inflation 


.338 


.343 


.551 


.161 


.164 


.243 


uuns 




. J02 


.180 


-.032 


.231 


-.033 


Polkas 


.145 


.292 


.757 


.290 


.459 


.436 


Parties 


.338 


.344 


.612 


.368 


.421 


.428 


Radishes 


.002 


.001 


.611 


.046 


.095 


.311 


Love 


.357 


.027 


.396 


.337 


.494 


.276 


Summer 


.326 


.291 


.652 


.205 


.318 


.384 


Alka-seltzer 


.044 


.082 


.332 


.149 


.332 


.353 


Average 


.244 


.222 


.567 


.192 


.366 


.356 



Table 7. Ksan salience scores i'or (1) Group A as determoned by pies, ranks, 
and SDS with corresponding numerical rankings and (2)Group B as 
determined by pieo, rankings, and LS with corresponding numerical ranks. 



Pies 



GROUP A 
Ranks 



SDS 



TRAITS 

Parents 334.47(1) 2.19(1) 3.34(2) 

Blood 266.32(6) 5.44(6) 4.14(4) 

Inflation 274.3(3) 4.70(5) 4.47(5) 

Guns 260.23(7) 8.20(9) 7.12(7) 

Polkas 184.10(8) 7.81(7) 8.07(8) 

Parties 287.97(4) 4.53(4) 5.88(6) 

Radishes 164.36(10) 8.34(10) 9.05(9) 

Love 325.26(2) 2.77(2) 3.27(1) 

Sumner 315.03(3) 2.95(3) 3.96(3) 

Alka-Seltz 172.40(9) 8.17(8) 9.56(10) 



Pies 



GROUP B 
Ranks- 



338.73 (1) 
271.57(6) 
267.03(7) 
287.43(5) 
211.83 (8) 
293.80(4) 
186.97(9) 
303.20(3) 
322.67(2) 
177.77(10) 



1.40(1) 
5.14(6) 
4.86 (5) 
7.74(8) 
7.51(7) 
4.69(4) 
8.30(9) 
3.52(3) 
3.19(2) 
8.59(10) 



LS 



2.69(1) 
7.41(8) 

3.52 (2) 
6.65(6) 
7.21(7) 
6.00(4) 
10.45(9) 
6.44(5) 
4.09(3) 

12.07(10) 



ERIC 



19 



BIBLIOGRAPHY 



Campbell, D.T. and B.W. Fisko 

Convergent and discriminant validation by the multitrait-multimethod matrix. 
Psychological Bulletin , 1959, 56, 81-105. 

Cartwright,D. and F. Harary 

Ambivalence and Indifference in generalizations of structural balance. 
Behavioral Science , 1972 . ??, 497-513. 

Duncan, O.D. 

Path Analysis: Sociological examples. IN H.M.Casfnor' (Ed.)> Sociological 
Methodology 1972 . San Francisco: Joseey-Bass, 1972, 36-82, 

Fishbein, M. and B. Raven. 

An operational definition of belief and attitude. Human Relations , 1962,15, 
35-44. 

Hays , W. 

Statistics for Psychologists . New York: Holt, Rinehart, & Winston, 1966. 

lleise, D.R. ^^roblems in path analysis and causal inference. In E.F. Borgatta 
(Ed.), Soc5>oloRical Methodology 1969 . San Francisco: Jossey-Bass, 1968, 38-73. 

Heise ,D.R. 

Separating reliability and stability in test-retest correlation. In H.M. 
Blalock (Ed.), Causal Models in the Social Sciences . Chicago: Aldine- 
Atherton, 1971,348-363. 

Hunter , J. E. 

Dynamic Sociometry. Unpublished manuscript: Dept. of Psychology, Michigan 
State University, East Lansing, 1974. 

Hylton, C. and W. Lashbrook 

Apathetic and noutral audiences: A computer simulation and validation. 
Speech Monographs > 1972,39,105-113. 

Jaccard,J., Weber, and J. Lundmark ? 

A multitrait-multimethod analysis of four attitude assessment procedures. 
J. of Experimental Social Psychology , 11,1975,149-155. 

Lashbrook, W.B. and La shbrook, V.J. 

The effects of message sideduess for apathetic and neutral encoders engaged 
in a countrorattitudinal task. Paper presented to the Annual Convention 
of the Speech Communication AGsn., Houston, 1975. 

Lashbrook, VJ.J. and T. Bush 

An empirical investigation of apathetic and neutral attitude response 
to manipulations of source credibility, use of evidence and language intensity. 
Paper presented to the \^estern Speech Communication ASSn., Albuquerque, 1973. 

Lashbrook, W.J. and J.Sullivan 

Apathetic and neutral audiences; More or Simulation and validation. Speech 
Monographs , 1973,40, 317-321. 

Lull, J.T. and P. Arnston 

Attitude pie: A departure from linear measurement. Paper presented to the 
Q Annual Convention of the International Communication Assn., New Orleans ,1974. 

ERIC 

20 



Osgood, C. E. , G.J. Suci, and P.H. Tannenbaum 

The Measurement of Meaning . Urbana,Ill.; University of Illinois Press, 1957. 

Sullivan, D. and W. Lashbrook 

A simulation and validation of computer adaptations to apathetic audiences. 
Paper presented to the Western Speech Assn. , Newport Beach, 1974. 

Torgerson, W. 

Theory and Methods of Scaling . New York: Wiley, 1958. 

Troyke,D. and W. Lashbrook 

Computer modeling for the speech communication classroom. Paper presented 
to the Annual Convention of the Speech Communication Assn. , New York, 1973. 

Wilejz, D.E. and J. A. Wiley 

The estimation of measurement error in panel data. In H.M. Blalock (Ed.), 
Causal Models in the Social Sciemces . Chicago: Aldine , 197.1,364-373. 

Wiley, J. A. and M.G. Wiley 

A note on correlated errors in repeated measurements. Sociological Methods 
and Research, 3, 1974, 172-188. 



ERIC 



21 



APPEiroiX A 



Attitude Questionnaire 



Please use the circles or "pies" on Page 3 to show vhat you think about the 
following attitude objects or concepts. Consider each circle as representative of 
the total or sum of all your feelings about the object or concept with which it 
corresponds. Think about each Item separately. It may be that a certain part of 
you reacts positively to the object or concept. At the same time there may be another 
part of you that reacts negatively to It. Indicate these separate feelings by 
drawing In "allces" on the pies to reflect the amount of both your positive and 
negative reactions. For example, consider this attitude object: 



In many ways ycu may react positively to this object. Then again there may be 
some ways In which this object conjurs up a negative reaction in your mind. In this 
care, then, you could respond to the object by drawing in slices on the pie so that 
they look something like this: 



Notice that you don*t have to fill In the v?holG "pie." Put a plus sign (+) 
In the slice you draw to represent the degree of your positive reaction to the object 
or concept. Put a minus sign (-) In the slice which shows your negative reaction* 
If these two slices do not completely fill the pie (as in the case above), simply 
leave the remaining slice unmarked. 

In this way you can use the pies to chow the streng;th of your reaction as well 
as the relative amount of positive and negative feelings you have. Let's say, for 
example, that you react very strongly to an object or concept and have nearly equal 
positive and negative feelings about it. You could show this reaction by filling 
in the pie scmetllng like this: 



ii< 



'Economy Cars 




\ 




\ 
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On the other hand, if you have neutral feelings about the object or concept 
out only a weak general reaction to it, you can divide up the pie to look something 
like this: 




If your reaction to any object or concept is complotely positive or completely 
negative, you can show this by putting in a plus or minus sign in the entire circle 
or any portion of it, depending upon the strength of your overall reaction: 




You have complete freedom to show precisely how you react to the attitude 
objects and concepts on the next page. Be sure to consider all your feelinps abi 
each item. Remember, the oize of the slices you draw represents the strength of 
your positive and negative reactions. 

Turn the page and begin. 



ERIC 



23 



-3- 




APPENDIX B 



Note that four of tho Semantic Differential scales are identical across traits. 
The fifth antonym pair peculiar to each trait i« reported below: 

Gf.vlng Blood 

THE CHOICE OF WHETHER OR NOT TO GIVE BLOOD MEANS A LOT TO ME. 
EVERYONE SHOULD DONATE BLOOD. 
I DON'T WANT TO GIVE BLOOD. 

THE IDEA OF GIVING BLOOD IS ON MY MIND A LOT. 
THE THOUGHT OF GIVING BLOOD MAKES ME SICK. 
Semantic Differential: pleasant-unpleasant 

Polkas 

WHEN I DANCE, I HATE TO POLKA. 

MORE PEOPLE OUGHT TO LEARN TO POLKA. 

POLKAS SHOULD NOT BE IGNORED. 

WE OUGHT TO GET RID OF POLKAS. 

POLKAS DON'T MEAN A THING TO HE. 

Semantic Differential: worthless -valuable 

Handguns 

I WOULD LIKE TO OWN A HANDGUN SOMEDAY. 

THE HANDGUN COOTROVERSY SHOULD BE A MORE IMPORTANT NATIONAL ISSUE. 

HANDGUNS ARE HUMANE. 

I NEVER THINK ABOUT »-i.t:r • -tj?-]?! . 

HANDGUNS f'l-OJT.D BE HADE "U r.oAL. 

Semantic n'- irforentlal: cruel-kinu. 

Making Love 

THERE ARE SOME THINGS ABOUT !■ l XHOUGtil Of r.AKING LOVE THA'i' ARE DISGUSTING. 
PEOPLE DON'T MAKE LOVE ENOUGH. 

SOMETIMES THE THOUGIiT OF MAKING LOVE SEEMS MORE BAD THAK 500D. 
S THE IDEA OF M.\KING LOVE IS NO MORE MEANINGFUL TO ME THAN ANY OTHER HUMAN BEHAVIOR. 
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I CAN GO FOR MYS WITHOUT THINKING ABOUT 1-lAKING LOVE WITH SOMEONE. 
Semantic DlffGrcntlal: awful-nlcc 

Smmncrt imc 

SUMMERTIME IS PURE JOY. 
I DON'T REALLY CARE ABOUT SUMMERTIME. 
THERE IS NOTHING BETTER THAN SUMMERTIME. 
I LIKE TO THINIC ABOUT SUMMERTIME A LOT. 

THERE'S A LOT OF THINGS ABOUT SUMMER THAT AREN'T SO GREAT. 
Semantic Differential: sed -happy 

Alka-Seltzcr 

I AUtOST NEVER THIMC ABOUT ALKA-SELTZER 

I CAN ONLY THINIC OF GOOD THINGS TO SAY ABOUT ALKA-SELTZER. 

JUST THE THOUGHT OF ALKA-SELTZER IS AIIFUL. 

IF I DR/vNK ALKA-SELTZER IT WOULD PROBABLY Mi\ICE ME SICK. 

ALICA-SELTZER HEi'.NS A LOT TO ME. 

Semantic Differential: nlcc-awful 

Parties 

I ALIIOST ALWAYS ENJOY llVbELF AT A PARTY. 

I DON'T CARE ABOUT ^ARTIES IN THE SLIGHTEST. 

I WANT TO GO TO TC V OF VCIES. 

I THINiC ABOUr PAKTift -■"li.Cr OVTEN. 

PART1.R!: ARE A DRAG. 

Semaur * Oiffcret)u7,al : unpleaj.'Ui!;.-pleusant 
Inflation 

THERE'S NO NEED TO THm AHOU- /.WPLAT" Oil. 

IT'S ALRIGHT IF THE RATE OF INFLATION KEEPS CLIMBIN'4. 



ErJc INFLATION IS EVERYBODY'S BUSINESS. 
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A NATION IS IN EAD SIIAPE WHEN INFLATION HITS ITS ECONOMY. 
PEOPLE SHOULD NOT BE PLEASED MITII INFLATION. 
Semantic Differential: pleasant -unpleasant 

Parents 

MY PARENTS ARE OFTEN INSENSITIVE TO THE NEEDS OF OTHERS . 
MY PARENTS ARE EXTREMELY IMPORTANT TO ME. 
MY PARENTS ARE VERY GOOD PEOPLE. 

THERE ARE REALLY ONLY A FEW POSITIVE THINGS TO SAY ABOUT MY PARENTS. 
MY PARENTS REALLY DON'T MATTER TO ME. 
Semantic Differential: kind-cruel 

Radishes 

RADISHES TASTE TERRIBLE. 
WE SHOULD THINIC MORE ABOLT! RADISHES . 
RADISHES WEAN ALMOST NOTHING TO ME . 
RADISHES ARE A PLEASANT ADDITION TO A MEAL. 
THERE'S NOTHING WRONG WITH RADISHES. 
Senantic Differential: pleasant -unpleasant 



ERIC 
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